Techniques for rendering advertisments with rich media

ABSTRACT

In one embodiment, an advertisement is matched to subject matter in a portion of rich media content, such as a digital video, Flash™ animation, etc. For example, during the playing of rich media content, it may be determined by audio recognition techniques that the content&#39;s subject matter matches or correlates with an advertisement. Rendering preferences associated with the advertisement are then determined. The rendering preferences may be used to determine how the advertisement should be rendered (i.e., displayed in association with the content). The advertisement is then served to a device. The advertisement is served such that it can be rendered relative to a time that the portion of media is being displayed on the device.

CROSS REFERENCES TO RELATED APPLICATIONS

This application claims priority from U.S. Provisional PatentApplications No. 60/733,874, entitled “Method and System forContextually Matching Advertisements with Rich Media Content”, filedNov. 7, 2005 and 60/784,415 entitled “Method and System for ContextuallyMatching advertisements with Rich Media Content”, filed Mar. 20, 2006,which are incorporated by reference in their entirety for all purposes.

BACKGROUND

Embodiments of the present invention generally relate to digital mediaand more specifically to displaying advertisements with rich mediacontent.

A user can perform a text search for content using a search engine. Whenthe search is matched to text content, the results are displayed on aweb page. The search results are typically static. For example, if auser was searching for certain web pages, the web pages and URLs wouldbe listed on the page and do not change.

Advertisements related to the content may then be placed in certainsections of the page. Because the content on the page is static, theadvertisements are matched to the search once. The placement of theadvertisement on the page may be optimized, such as placing theadvertisements at the beginning of the results. However, because thecontent on the web page is static, there is no need to match theadvertisements to content that changes over a period of time. It isassumed that once the search is finished, the content remains the same.

SUMMARY

Embodiments of the present invention generally relate to servingadvertisements with rich media content.

In one embodiment, an advertisement is matched to subject matter in aportion of rich media content, such as a digital video, Flash™animation, etc. For example, during the playing of rich media content,it may be determined by audio recognition techniques that the content'ssubject matter matches or correlates with an advertisement. Renderingpreferences associated with the advertisement are then determined. Therendering preferences may be used to determine how the advertisementshould be rendered (i.e., displayed in association with the content).

The advertisement is then served to a device. The advertisement isserved such that it can be rendered relative to a time that the portionof media is being displayed on the device. For example, theadvertisement may be rendered on the device according to an advertisingplan paid for a by a sponsor. The advertising plan can include criteriafor matching the advertisement to the content, a manner of displayingthe ad (size, position, ad content, ad animation, etc.), or otherfactors. In one embodiment, the advertisement may be displayed seriallywith the portion of rich media content, in parallel with the portion ofrich media content, injected into the rich media content, etc.

A further understanding of the nature and the advantages of theinventions disclosed herein may be realized by reference of theremaining portions of the specification and the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts a simplified system for serving advertisements with richmedia content according to one embodiment of the present invention.

FIG. 2 depicts a more detailed example of the system shown in FIG. 1according to one embodiment of the present invention.

FIG. 3A depicts a video example of rendering advertisements seriallyaccording to one embodiment of the present invention.

FIG. 3B shows an audio example of rendering advertisements serially inrich media content that includes audio according to embodiments of thepresent invention.

FIG. 4A depicts a video example of rendering advertisements in parallelaccording to embodiments of the present invention.

FIG. 4B depicts an audio example of rendering advertisements in parallelaccording to embodiments of the present invention.

FIG. 5 depicts a system for initiating serving of advertisementsaccording to one embodiment of the present invention.

FIG. 6 shows an example interface according to one embodiment.

FIG. 7 shows an example of how content may relate to keywords purchasedby advertisers according to one embodiment.

FIG. 8 shows an example of the scoring system according to oneembodiment.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

System

FIG. 1 depicts a simplified system 100 for serving advertisements withrich media content according to one embodiment of the present invention.As shown, an engine 102, user device 104, advertiser system 106, andcontent owner system 108 are provided.

Engine 102 may be any device/system that provides serving ofadvertisements to user device 104. In one embodiment, engine 102correlates advertisements to subject matter associated with rich mediacontent. Accordingly, an advertisement that correlates to the subjectmatter associated with the portion of rich media content may be servedsuch that it can be rendered on user device 104 relative to the portionof rich media content. Different methods may be used to correlate ormatch advertisements to portions of the rich media content.

Advertiser system 106 provides advertisements from advertisementdatabase 112. Advertisements may be any content. For example,advertisements may include information about the advertiser, such as theadvertiser's products, services, etc. Advertisements include but are notlimited to elements possessing text, graphics, audio, video, animation,special effects, and/or user interactivity features, uniform resourcelocators (URLs), presentations, targeted content categories, etc. Insome applications, audio-only or image-only advertisements may be used.

Advertisements may include non-paid recommendations to otherlinks/content within the site or to other sites. The advertisement mayalso be data from the publisher (other links and content from them) ordata from a servicer of engine (e.g., from its own data sources (such asfrom crawling the web)), or some other 3^(rd) party data sources. Theadvertisement may also include coupons, maps, ticket purchaseinformation, or any other information.

An advertisement may be broken into ad units. An ad unit may be a subsetof a larger advertisement. For example, an advertiser may provide amatrix of ad units. Each ad unit may be associated with a concept. Thead units may be selected individually to form an advertisement. Thus,advertiser system 106 is not restricted to just serving an entireadvertisement. Rather, the most relevant pieces of the advertisement maybe selected from the matrix of ad units.

The ad units may perform different functions. Instead of just relayinginformation, different actions may be facilitated. For example, an adunit may include a widget that collects user information, such as emailaddress or phone number. The advertiser may then contact the user laterwith additional info about its products/services.

An ad unit may also include a widget that stores a history of ads. Theuser may use this widget to rewind to any of the previously shown ads,fast forward and see ads yet to be shown, show a screen containingthumbnails of a certain number of ads such that a user can choose whichone to play, etc.

An ad unit may include a widget that allows users to send the ad toothers. This facilitates viral spreading of the ad. For example, theuser may use an address book to select users to forward the ad to.Further, an ad unit, when it is replaced by another ad unit, may beminimized into a small widget that allows the user to retrieve the ad,send to others, etc.

Different ways of creating an ad unit may also be appreciated. An adunit may be created by applying a template on existing static ad unitsand turning them into video that may serve as pre/mid/post-roll. An adunit may be created by taking a static ad and augmenting the unit withan advertiser-specified message dependent on context and keywords.

Advertisements will be described in the disclosure, but it will beunderstood that an advertisement may be any of the ad units as describedabove. Also, the advertisement may be a single ad unit or any number ofa combination of ad units.

Advertiser system 106 provides advertisements to engine 102. Engine 102may then determine when to serve advertisements from advertisementcontent 112 to user device 104. This process will be described in moredetail below.

Content owner system 108 provides content stored in content database 114to engine 102 and user device 104. The content includes rich mediacontent. Rich media content may include but is not limited to contentthat possesses elements of audio, video, animation, special effects,and/or user interactivity features. For example, the rich media contentmay be a streaming video, a stock ticker that continually updates, apre-recorded web cast, a movie, Flash™ animation, slide show, or otherpresentation. The rich media content may be provided through a web pageor through any other methods, such as streaming video, streaming audio,pod casts, etc.

Rich media content may be digital media that is dynamic. This may bedifferent from non-rich media content, which may include standardimages, text links, and search engine advertising. The non-rich mediamay be static over time while rich media content may change over time.The rich media content may also include user interaction but does nothave to.

User device 104 may be any device. For example, user device 104 includesa computer, laptop computer, personal digital assistant (PDA), cellulartelephone, set-top box and display device, digital music player, etc.User device 104 includes a display 110 and a speaker (not shown) thatmay be used to render content and/or advertisements.

Advertisements may be served from engine 102 to user device 104. Userdevice 104 then can render the advertisements. Rendering may include thedisplaying, playing, etc. of rich media content. For example, video andaudio may be played where video is displayed on display 110 and audio isplayed through a speaker (not shown). Also, text may be displayed ondisplay 110. Thus, rendering may be any output of rich media content onuser device 102.

In one embodiment, the advertisements are correlated to a portion of therich media content. The advertisement can then be displayed relative tothat portion in time. For example, the advertisement may be displayed inserial, parallel, or be injected into the rich media content. Thedifferent ways of rendering the advertisements relative to the richmedia content will be described in more detail below.

Correlation

FIG. 2 depicts a more detailed example of system 100 according to oneembodiment of the present invention. As shown, engine 102 includes acorrelation engine 202, a rendering formatter 204, an ad server 206, acontent database 208, an ad database 210, a recognition engine 212,correlation assistant 214, an optimization engine 216 and advertiser website 218.

Correlation engine 202 receives advertisements from ad database 208 andcontent from content database 208. The advertisements and content mayhave previously received from content owners and advertisers.

Correlation engine 202 is configured to determine an advertisement thatcorrelates to subject matter associated with a portion of the rich mediacontent. For example, at a certain time, period of time, or multipleinstances of times, an advertisement may be correlated to subject matterin the rich media content. For example, an advertisement may beassociated with a keyword. When that keyword is used in the rich media,correlation engine 202 correlates the advertisement to a portion of richmedia content in which the keyword is used.

Correlation engine 202, when determining the advertisement, maydetermine one or more ad units that correlate to the subject matter. Forexample, based on one or more keywords, ad units from the ad matrix aredetermined. The ad units are then combined into an advertisement that iscorrelated to the subject matter. One example of this is BMW may providea general ad unit for their logo and have a different ad unit fordifferent models, such as the 330 model, 530 model, etc. The logo unitand each of the model units can be combined at runtime based on thecontext of the content. If the content talks about the 330 model thenthe logo and the 330 ad units may be combined and presented to the user.

Correlation engine 202 may also use user information to correlateadvertisements to subject matter associated with a portion of the richmedia content. User information may be any user information that isassociated with a user of user device 104. For example, user informationmay include information about the behavior of a user. An accountbehavioral analysis of user information, such as from cookies, fromaccount information, etc. may be used, in addition to contextualanalysis of the content. This may help correlate which advertisementsmay be more relevant to the user based on the previous behavior of theuser. Other user information may also include demographic data, recentsites visited by the user, and user actions such as mouse movementduring presentation of an ad, etc. In one example, if the user hadpreviously visited web sites for a certain advertiser, then it may bemore effective to send the user an advertisement from that advertiser.

Correlation recognition detection techniques may be used to determinethat the advertisement is correlated to the portion of rich mediacontent. For example, keywords may be detected in the rich media usingaudio recognition. Audio recognition may include speech recognition,music detection on music portions, sound effect detection on soundeffects, etc. Other techniques for keyword detection can include usingpreset word tags or indicators in the rich media content. Imagerecognition can be used on visual portions of the rich media content.For example, optical character recognition (OCR), facial recognition,object matching, etc. Other recognition techniques can be employed. Forexample, any suitable way of determining the content of rich media canbe used to correlate a portion of the rich media content to anadvertisement.

Recognition engine 212 receives rich media content that may be accessedby a user and uses correlation recognition detection techniques torecognize the content. In one embodiment, recognition engine 212 usingaudio recognition techniques, such as speech recognition, whereby theaudio within the content would be recognized. In another embodiment,recognition engine 212 may be a video or visual analysis engine thatgenerates data regarding the visual content of the rich media. Inanother embodiment, it could be natural language processing engine. Inyet another embodiment, it may be an indexing engine that culls andprocesses textual meta-data associated with the content. Other audiorecognition techniques may also be appreciated.

Recognition engine 212 is a processor that outputs relevant tags aboutthe content that the rest of system 100 may use to match appropriate adsat the appropriate time when the content is accessed and played. Forexample, the information may be keywords recognized from audio in thecontent. Recognition engine 212 may generate a unique ID for each pieceof content. In one embodiment, recognition engine 212 transcribes theaudio within the content into text and related concepts. In the video orvisual recognition embodiment, meta-data about the visual content isgenerated or culled from the content itself. In the textual meta-dataembodiment, the meta-data is culled from the content and processed. Eachconcept, word, or meta-data may be time-stamped and stored in storage208. It may be stored in a hash table, index, or database. This providesan index of information recognized from the content.

In some cases, recognition engine 212 may not definitively recognize aword. When uncertainty exists, such as with speech recognition or visualanalysis, a lattice of the concepts, words, or meta-data are stored withassociated probabilities. The lattice contains alternatives to the bestchoice when the probability associated with the best choice is low.Similar words may be determined and included in the lattice. Also,alternative words for words that are typically hard to recognize mayalso be included in the lattice.

The recognition may be performed in non real-time or in real-time. Innon real-time, content owners submit their content to recognition engine212 prior to an audience viewing the content. In the real-timeembodiment, the content would be recognized as it is being accessed andplayed by users.

In determining when an advertisement correlates to subject matterassociated with a portion of rich media content, a time to render theadvertisement relative to the content that maximizes one or morecriteria may be determined. For example, correlation engine 202 maydetermine a sequence of advertisements that maximize revenue. Theseadvertisements are time aligned to correlate to the subject matter tomaximize the revenue.

Advertisers may buy correlation information, such as keywords, phrasesor concepts, either through a bidding process or some other means, andsubmit their ads and related information to correlation engine 202though correlation assistant 214. Keywords may be words that can be usedto match information in the content. The phrases may be any combinationof words and other information, such as symbols, images, etc. Theconcepts may be a conceptual idea of something. For example, if aportion of rich media talks about Lebron James, this can beconceptualized to basketball, and we can match advertisements tobasketball to the rich media even if for some reason the exact term“basketball” is not mentioned. The related information includes but isnot limited to URLs, presentations of ads, targeted content categories,etc. to be associated with the ad space or inventory they have bought.The advertiser can also specify anti-keywords, phrases, or concepts. Ananti-keyword is a keyword or phrase that an advertiser chooses such thatif that keyword or phrases is recognized in the rich media content, theadvertiser's ad would not be shown, even if there is a keyword/phrasematch.

The selection of normal/anti-keywords, normal/anti-phrases, andnormal/anti-concepts is useful for ad matching performance. For example,if General Motors purchases the keyword “General Motors” for its ad, thead may very well be matched to a video discussing a General Motorslayoff, if the anti-keyword “layoff∓ is not selected. To prevent thesematches, correlation assistant 214 determines appropriate content foradvertiser's ads. Once advertisers enter their desired keywords,phrases, or concepts, correlation assistant 214 may present other textcorrelated to those keywords. The other test may be based on analysis ofhistoric content in the recognized content. Also, the related text mayalso be taken from external knowledge sources and databases. Forexample, a spider may search the web for popular news stories. The newsstories may be indexed and words may be determined that are found in thestories. If the word “General Motors” occurs frequently near the word“layoff” in the index, correlation assistant 214 may suggest the term“layoff” when “General Motors” is chosen. The advertiser can thenspecify whether each of the desired keywords or text fragments suggestedcorrelation assistant 214 is positive, neutral, or negative to thecontent they would like to target. In this way, the advertiser canbetter refine their selection. A list of anti-keywords may then bedetermined and stored in storage 210.

Also, the keywords that an advertiser wants to bid on may also bedetermined using correlation assistant 214. Suggestions as to whichkeywords may be associated with an advertiser may be provided. The morepopular keywords may then be bid on by the advertiser. By showing howpopular a keyword is, higher bids may be received or required. Also,when a keyword is bid on by an advertiser. Correlation assistant 214 maydisplay similar keywords for an advertiser to choose from. This may givean advertiser more keywords to bid on or even better keywords that mayresult in better matches.

Advertisers may also specify other associations for their ads. Suchassociations may include but are not limited to keyword/anti-keyword,phrase/anti-phrase, concept/anti-concept, and domaincategory/anti-category. A category may refer to sports, news, business,entertainment, etc.

Correlation assistant 214 may generate a unique ID for each adsubmitted. These ads and related information would be stored in storage214 as an index or other data structure. In one embodiment, each ad unitmay be identified. For example, a matrix identification system may beused to identify different ad units for an advertiser.

The data and meta-data (for example, spoken words and concepts) of theindexed and time-stamped content are stored in storage 208. Correlationengine 202 may relate a unique content ID with a time series of keywordsand concepts (that advertisers have purchased), and in turn, relate thekeywords and concepts to ads submitted by advertisers.

FIG. 7 shows an example of how content may relate to keywords purchasedby advertisers according to one embodiment. A recognition of content 702is shown. This is a text version of rich media content. For example, anewscast has been translated from speech to text using speechrecognition techniques.

The ads may be correlated to content in different ways. In oneembodiment, keywords may be associated with each ad. Content may besearched to determine if the content includes the keywords. Then the admay be correlated to the content. For example, an index of keywords maybe generated for each piece of content. The correlation may be performedfor all ads and a series of ads may be matched to different content instorage 208.

In another embodiment, keywords in a piece of content may be determined.Different ads that correlate to the content may then be determined. Forexample, for each keyword determined, one or more ads that areassociated with the keyword are determined. The ads to display may thenbe selected.

The correlation may be performed in non real-time or in real-time. Inthe non-real time embodiment, correlation engine 202 may perform thecorrelation at regular intervals.

In one example of correlation in the non real-time embodiment, for eachad, correlation engine 202 finds candidate content that may be relevant.This is done by searching for content in the index to match thekeywords, categories, and concepts associated with the ad to informationin the content. Correlation engine 202 may perform the search not onlyon the top choices (e.g., keywords) generated by recognition engine 212,but on the entire lattice of possible alternatives. The content indexmay also include a long tail of words, especially proper nouns such asnames and foreign entities. This long tail includes many proper nounssuch as names and foreign entities. When an out-of-vocabulary keyword isassociated with an ad, that word is converted to other in-vocabularywords with similar phonetic make-up.

For each piece of candidate content associated with an ad, correlationengine 202 determines candidate times where the content may be relevantto the ad. Correlation engine 202 locates the times where the keywordsand concepts match. For each candidate time, correlation engine 202creates an “ad anchor” holding the score for the match. The score may bea linear combination of the following weights:

-   1. Probability of the keyword/concept match pulled from the    recognition lattice.-   2. Concentration of the match—the more keywords/concepts for the ad    matches near the time, the higher the score. One embodiment of this    score may be a count of the number of matches within a certain    window of the current time.-   3. Position of match relative to other matches—the more likely that    the current time starts a segment on the keywords/concepts, the    higher the score. One embodiment of this score may be the inverse of    a count of the number matches before a certain amount of time.-   4. Historical performance—the prior performance of an ad. For    example, if an ad previously placed resulted in a user selection of    the ad, the ad may be considered a better match.-   5. Price of the ad placement—the price of the bid may be taken into    account to determine the match. For example, higher bids for the ad    may be considered better matches. Also, higher bids may be placed in    a separate area of the display screen, etc.-   6. Geo-location match—the area where the content may be displayed    may be considered in matching the ad to content.-   7. Demographic—a classification of a user, such as age bands, social    class bands, and gender.-   8. Behavioral—the actions or reactions of an object or organism,    usually in relation to the environment.-   9. Psychographic profile—any attributes relating to personality,    values, attitudes, interests, or lifestyles.-   10. Past performance—the past performance of an ad.

For each piece of content, correlation engine 202 may prune away the lowscoring anchors. For example, a threshold may be used where anchorsbelow the threshold are not considered. For each remaining anchor,correlation engine 202 extends the segment for the ad to the minimumtime for the ad. This is either a global minimum or a setting configuredby the advertiser or content owner. For example, if the ad is a logo,the logo may be displayed for the minimum amount of time.

For each pair of overlapping ads, correlation engine 202 disambiguatesthe overlapping ads by selecting the ad with the higher score. Remainingempty space may be filled by extending the previous ad, filling it withpruned ads, or by filling with ads that score generically well. Also, adunits may be chosen to fill in the space. If ads are in 5 secondintervals, the best matching 5 second intervals may be chosen dependingon how much empty space there is to fill.

In the real time embodiment, recognition engine 212 may determinekeywords from the content. As recognition engine 212 produces astreaming lattice of keywords or concepts, a rolling window of thelattice is used to look up potentially relevant ads. Whenever a possiblerelevant ad is found, the ad is scored in the same way as in thenon-real time case. Correlation engine 212 makes an immediate decisionon whether the ad is appropriate. A hard threshold may be used to makethe determination. The value of the threshold is determinedexperimentally to optimize relevance and minimize false acceptances.

As shown in FIG. 7, advertisers 704 have keywords 706 associated withads. Also, association rules 708 are provided that guide when tocorrelate an ad to the keyword.

Advertiser 704-1 is the Major League Baseball and has specified thekeywords “Red Sox” and “Fenway”. The ad may be correlated to the keywordwhen the two words are found in content proximate to each other. Forexample, the words may have to be within a certain number of words inthe content.

Advertiser 704-2 is Nike and the keyword “Superbowl” is associated withit. The ad for Nike may be shown when the keyword Superbowl is spoken incontent.

Advertiser 704-3 is BMW and the keyword “Luxury Automobile” isassociated with it. Also, an anti-keyword shown as NOT “pollution” isalso associated with BMW. In this case, when the word luxury automobileis spoken without the word pollution in proximity, then the BMW ad maybe matched to the content.

As shown in content 702, the keywords are found for all threeadvertisers. The ads may be correlated to the content at the time thewords are spoken in the content. As shown in FIG. 8, as the words arespoken, as advertisement for each advertiser 704 is shown.

In one embodiment, a logo 710 for advertisers 704 is shown and displayedin parallel with the content. Other display options may be used and aredescribed herein. A uniform resource locator (URL) 712 may be associatedwith the ads. For example, when the ads are selected or clicked on, awindow may be opened and redirected to the URL.

The information shown in FIG. 7 may be stored as a table or tablesrelating content and content-related information, time series to displayads and time series-related information, and characterizations of theads to be displayed (ad ID, presentation of ad, URL of ad, etc.). Thisthen in effect relates ads to content. In the real-time embodiment, thecontent is processed by the recognition engine and matched with ads asthe content is being played to a user.

FIG. 8 shows an example of the scoring system according to oneembodiment. As shown, recognized text for a piece of content is shown inblock 802. In this example, Motorola owns the keyword “razr”, Verizonowns the keyword “e.v.d.o.”, T-Mobile owns the keyword “g.p.r.s.”, andSeagate owns the keyword “megabyte diskdrive”. Correlation engine 202recognizes the content with the exception of “megapixel camera” beingmisrecognized as “megabyte diskdrive”. In this case, the spoken word wasrecognized as the wrong word, which may happen from time to timedepending on the recognition engine that is used. Note that “razr” showsup twice in the transcript hence is scored twice.

The scores in table 804 may correspond to any of the 10 weightsdescribed above. More or fewer weights may be included in table 804. Asshown, five weights are being used. All the recognition scores aresimilar except the misrecognized phrase gets a much lower score. Therecognition score may be related to the confidence that the word wasproperly recognized. The match concentration score is highest for “razr”because the word shows up twice near each other, whereas the otherkeywords only show up once.

The match position scores are similar except for the second occurrencefor “razr” since initial positions are preferred. In this example, thead is new so no score is found for any past performance scores. However,if an ad was previously placed and was considered a good placement, thepast performance score may be high. For example, if an ad is selected bya user when it was placed, it may receive a good performance score.Also, if the ad was selected and then the user purchased something fromthe advertiser, then the performance score may be even higher.

The ad cost/price also shows that Motorola has bid the most for thekeyword razr. Accordingly, it has the highest score for the adcost/price. However, if Verizon had bid more for e.v.d.o, then it mighthave skewed the final weight in its favor and thus Verizon's ad may beplaced.

Based on the scores, Motorola wins the ad placement in this segment andthe ad is shown right after the first occurrence of “razr”. If, however,the placement is made, and nobody selects or clicks on the Motorola ad,but on the other hand, the Verizon ad has found success elsewhere withthe keyword “e.v.d.o.”, the next iteration of the scoring will likelyaward the placement to Verizon, increasing the potential revenue fromthe placement. Optimizer engine 216 may receive previous performanceinformation from advertiser website 218. For example, clickthroughinformation, purchase information, and billing information, and otheruser interaction information may be received. This information may beused to determine a past performance score.

Rendering

In determining when an advertisement correlates to subject matterassociated with a portion of rich media content, a time to render theadvertisement relative to the content that maximizes one or morecriteria may be determined. For example, correlation engine 202 maydetermine a sequence of advertisements that maximize revenue. Theseadvertisements are time aligned to correlate to the subject matter tomaximize the revenue.

Rendering formatter 204 then determines how the advertisement should berendered relative to a time the portion of the rich media content isbeing played. For example, rendering formatter 204 can determine that anadvertisement should be rendered serially relative to the portion ofrich media content, in parallel to the portion of rich media content, orinjected into the rich media content. In other embodiments, theadvertisements may be stored for later display by a user. For example,at any time, the stored advertisements may be displayed even if the richmedia content is not being played. In one example, the advertisementsmay be stored in a folder. An icon may be inserted in display 110, whichcan be selected and used to cause display of the stored advertisements.

When an advertisement is rendered serially relative to a time theportion of the rich media content is being played, it can be renderedpost-roll or pre-roll. Post-roll is after the time the portion of richmedia content is displayed. For example, the portion of the rich mediacontent may be played and then the advertisement may be rendered afterthe portion is played. In one example, a media player may be playing therich media content. When the portion of rich media is finished playing,the advertisement may be rendered using the player. Thus, a user canonly watch the rich media content or the advertisement. In otherembodiments, the advertisement may be rendered in a different area ofdisplay 110 but the playing of the rich media content is stopped whilethe advertisement is being played.

In the pre-roll case, the advertisement may be rendered before theportion of rich media content is played. Accordingly, in this case,either the advertisement or the portion of rich media content is beingrendered at one time. A variation on this case allows the advertisementto begin playing before the rich content and to continue playing with(i.e., overlap with) the content after the content starts playing.Similarly, a portion of post-roll advertising may overlap with the endof the rich media content.

In the parallel case, the advertisement may be rendered at substantiallythe same time the portion of rich media content is being played. Itshould be noted that at any time during the portion of rich mediacontent being played, the advertisement may be rendered. Theadvertisement does not have to be rendered for the entire time theportion of rich media content is being played. Also, the advertisementmay be rendered at other times when the portion of rich media content isnot being played. For example, the advertisement may continue to berendered after the portion of rich media content has finished beingplayed.

For the parallel case, the advertisement may be rendered in a separateportion of display 110. For example, in a different window. Also, othermethods may be contemplated, such as the advertisement is rendered in acut out section of a window playing the rich media content, in a pop upwindow, overlaid on the rich media content, as audio, etc.

The advertisement may also be injected into the portion of rich mediacontent. For example, as the portion of rich media content is beingplayed, the advertisement may be rendered in the rich media content.

Rendering formatter 204 may receive user preferences, content providerpreferences, and/or advertiser preferences and use them to determine howthe advertisement should be displayed. Although these preferences arediscussed, it will be understood that any other preferences may be used.

User preferences may be any settings that are associated with a user ofuser device 104. For example, users may prefer to view advertisements incertain ways. Also, user preferences may indicate a type of user device104. This may cause different rendering preferences to be used. Forexample, if the user is using a smaller user device 104, such as a PDA,then advertisements may be served serially. This is because a display110 on the PDA may be too small to display advertisements in parallel.However, if a user is using a user device with a larger display, such asa laptop computer, then advertisements may be served in parallel becausethe display is larger and may be able to accommodate displaying both theadvertisement and rich media content at the same time.

Content provider preferences may be preferences specified by the contentprovider. For example, a content owner may only want advertisementsshown in parallel because it does not want its content interrupted. Somekinds of content may be more effective if the advertisements are shownin parallel such as movies. However, some content, such as sportingevents, may be better suited for showing advertisements serially.

Advertiser preferences are preferences that may be associated with anadvertisement. An advertiser may set these preferences; however, otherentities, such as an owner of engine 102, may also set thesepreferences. The advertiser preferences may be optimal ways of renderingthe advertisement, such as positioning of the advertisement relative tothe content, sizing the ad, animating the ad, etc. Any other factorsthat affect presentation time or effect upon a viewer can be included inadvertiser preferences and can become part of a payment program oradvertising plan. For example, sponsors that pay more can have their adspresented multiple times over other sponsors who have not paid as much.This can work where multiple sponsors desire that their ads be shownduring the same content. Priority can be given to preferred (e.g.,higher-paying) sponsors so that the preferred sponsor ads are displayedmore often, in place of, or more prominently than non-preferredsponsors.

Advertising plans can include consideration for the amount of time thatan ad is relevant. Preferred sponsors can have their ads presentedduring the longer relevant intervals while non-preferred ads areallocated to the shorter intervals. If the correlation engine issophisticated enough to determine qualities such as “impact” (e.g.,dramatic or impressionable impact to a viewer) then preferred sponsorads can be given priority for the dramatic correlations. A dramaticcorrelation can be, for example, a hurricane report or other impendingweather condition correlating with ads for sales of provisions, survivalor safety equipment. Another type of dramatic correlation can be anannouncement that an entertainer is going on tour with a correlation toan ad for ticket purchases. Many other types of correlations arepossible and they can be ranked and matched to sponsors in various waysto achieve optimized revenue models for a company operating theadvertising server or acting as a management or sales entity for such asystem.

There may be rules that specify which preferences are used ifpreferences conflict. For example, the advertiser preferences may trumpany conflicts with user preferences or content owner preferences. Also,the preferences may indicate that the advertisement should be moved inspace. For example, if the keyword used is “BMW” but the advertisementis for a Cadillac, BMW may not a Cadillac advertisement so close to whenit is being mentioned. Thus, the Cadillac ad may be moved away from therich media content being played or it may be delayed until a later timewhen the rich media content is not being mentioned anymore.

Once rendering formatter 204 determines rendering preferences, ad server206 is configured to serve the advertisement. The advertisement may beserved as a function of time, serially or in parallel or injected, interms of presentation vis a vis the rich media content. In serving theadvertisement, ad server 206 sends it to user device 104 for display.The serving may occur substantially at a time when it should be renderedwith the rich media content. Also, the serving may occur prior to whenthe advertisement should be rendered. For example, the advertisement maybe preloaded onto user device 104. User device 104 can then render theadvertisement when it is appropriate according to the renderingpreferences.

Also, in other embodiments, ad server 206 may serve the advertisementfor storage in user device 104. For example, the advertisement may bestored and at a later time rendered. In one example, an icon may bedisplayed in display 110 and the user can select the icon causingdisplay of the advertisement.

FIG. 3A depicts a video example of rendering advertisements seriallyaccording to one embodiment of the present invention. As shown,advertisement 302 is rendered and video 304 is played in display 110. Anadvertisement 302-1 may be shown pre-roll to video 304. In one example,the video may be talking about a certain kind of car, such as thenewscaster shown is talking about an automobile. In this case, anadvertisement for a company that has purchased the keyword “automobile”may be shown before video 304. For example, every time the wordautomobile is used, an advertisement for a BMW car is shown. Aftershowing the advertisement, video 304 is shown. The content may continueuntil a second advertisement 302-2 is shown.

Second advertisement 302-2 may be shown post-roll or pre-roll to eithervideo 304-1 or video 304-2. For example, if advertisement 302-2 is shownpost-roll, then subject matter in a portion of rich media content 304-2may be correlated to advertisement 302-2. After video 304-1 is shown,advertisement 302-2 is shown. If advertisement 302-2 is shown pre-roll,then subject matter in video 304-2 is correlated to advertisement 302-2.Advertisement 302-2 is shown before video 304-2 in this case.

FIG. 3B shows an audio example of rendering advertisements seriallyaccording to embodiments of the present invention. As shown,advertisement 306-1 may be rendered before audio 308-1 is played. In oneembodiment, advertisements 306-1 may be displayed visually. Also, itwill be recognized that advertisement 306-1 may be an audioadvertisement such that it can match the content that is being played.Also, a combination of video and audio advertisements 306-1 may beprovided. This may be set by the preferences determined in renderingformatter 204.

After rendering of advertisements 306-1, audio 308-1 is played. A secondadvertisement 306-2 is then played in between audio 308-1 and audio308-2. Advertising 306-2 may be correlated to subject matter in eitheraudio 308-1 or audio 308-2 and played post-roll or pre-roll.

FIG. 4A depicts a video example of rendering advertisements in parallelaccording to embodiments of the present invention. As shown,advertisements 404 are rendered in parallel to video 402. As the videois being displayed, advertisements 404 may be rendered. In oneembodiment, advertisements 404 may be displayed along with video 402. Inone embodiment, each advertisement 404-1-404-5 may be a differentadvertisement. However, it will be understood that advertisements maypersist across multiple frames of video 402. For example, during theplaying of video 402-1 to 402-3, subject matter may be correlated to anadvertisement. This advertisement may be displayed in advertisements404-1 to 404-3.

At another time during the playing of video 402-4 and 402-5, a differentadvertisement may be correlated to subject matter in the video. Thus, adifferent advertisement is displayed for advertisements 404-4 and 404-5.

The advertisements displayed may be of a form that does not interferewith rendering of the video. For example, if the video includes audio,then a text advertisement may be shown. This does not interfere with theplaying of audio of the video. Also, the advertisement may include videowithout sound such that it does not interfere with the audio of thevideo.

FIG. 4B depicts an audio example of rendering advertisements in parallelaccording to embodiments of the present invention. As audio 406 isrendered, advertisements 408 are rendered. In one embodiment,advertisements 408 are rendered in form that does not interfere with theaudio. For example, the advertisements may be rendered as video withoutaudio or text.

Advertisements 408 may be associated with subject matter in the audiofor audio 406. As audio is being rendered, advertisements 408 may berendered in parallel.

Serving Ads

FIG. 5 depicts a system 500 for initiating serving of advertisementsaccording to one embodiment of the present invention. A content server504 is used to serve content to user device 104. For example, contentserver 504 may receive content from content owners 108 and send it touser device 104.

In one embodiment, content server 504 may serve a web page to userdevice 104 for display in display 110. When the rich media content isserved to user device 104, content server 504 contacts engine 102 tonotify engine 102 that content is being served. In one embodiment,content server 504 includes software code that enables it to connectwith engine 102. For example, software code may be embedded in a webpage. Also, the code may be integrated in the rich media content beingserved. It will be understood that there may not be a need to integrateany code and it depends on the functions of the content server 504 beingused.

Content server 504 contacts ad server 206 with information needed byengine 102 to serve advertisements with rich media content. For example,the information may include an identifier for the content being served,other content-related data, a time of the content as it is being played,user interaction data, an IP address of user device 104, any otherrouting information, and any other data relating to user device 104,content server 504, etc.

An identifier for the content being served may be converted into acontent ID that is recognized by ad server 206. With a content ID,correlation engine 202 can then determine advertisements to correlate tothe rich media content. For example, different advertisements may becorrelated to content associated with the content ID previously. Thecorrelating may have been performed when a content provider uploaded thecontent to engine 102. The advertisements that are correlated to therich media content are then stored with the content ID. In otherembodiments, the correlation may be performed dynamically as content isbeing displayed at user device 104.

A user information determiner 508 may be used to determine informationabout the user. As discussed above, the user information may be anyinformation associated with the user and may be used to correlateadvertisements to the content. In one embodiment, user informationdeterminer 508 may interact with user device 104 to determine the userinformation. For example, user device 104 may send a cookie toinformation determiner 508.

A geo-location server 502 may be used to determine any locationinformation that might be useful in serving advertisements. For example,an IP address may be converted into a physical geographical location ofthe user for user device 104. This may be used to serve advertisementsin a more relevant fashion, such as different ads are served based onthe location, different languages are used, etc.

Correlation engine 202 may use any of the above information to performthe correlation. Once the advertisements are determined by correlationengine 202, rendering formatter 204 determines how to render thecontent. Preferences described above may be used to determine how torender the content and advertisement.

Ad server 206 may then use the geographic location or IP address toserve the advertisement to user device 104.

In one example, if BMW has purchased/associated a keyword “automobile”for their advertisement, and rich media content includes the keyword“automobile” as it is rendered, the BMW ad may be served. It may beserved and rendered in parallel, near, around, in, or on the rich mediacontent as the keyword “automobile” is being spoken. In another example,the BMW ad may be rendered serially before and after the keyword“automobile” is played.

In another example, audio ads may be serially matched with pod casts(audio programs from the Internet) where the advertisements are placedin front, in the middle, or after the audio content. In addition, textor video ads may be displayed in parallel with the audio.

A user can interact with the advertisements served. For example, theuser may launch events from user device 104. In one embodiment, a usermay select the advertisement and launch another window with a website ofthe advertiser's choosing. In this example, when a user interacts withthe advertisement, software code embedded (if there is any) in theadvertisement sends relevant data to ad server 206. Ad server 206 canthen contact a billing and click-through server and update billing datato recognize that the advertisement has been selected. The data relatingto the user interaction with the advertisements may be fed back tocorrelation engine 202 to further optimize matching of advertisementswith rich media content.

Other features may also be provided as a user interacts with theadvertisements served. FIG. 6 shows an example interface 600 accordingto one embodiment. Rich media content may be displayed in media player602. Also, an advertisement is being rendered in section 604.

The advertisement may be a video and/or an image of an advertisement. Auser may also trigger actions by interacting with interface 600. Forexample, the rich media content may be played in player 602. An image ofthe advertisement is shown in section 604 (in other examples, a videomay be playing without sound). The advertisement, however, may also be avideo advertisement. Because playing two videos at once may be confusingto the user, a trigger is used to determine when to play theadvertisement and/or rich media content. For example, a mouse-overfeature is provided. When a mouse is moved over advertisement 604, theadvertisement may start playing and the rich media content is stopped.This allows the user to choose when to play the advertisement or richmedia content. If the user is interested in an advertisement that isshown, the user can choose to view the advertisement. When the user isfinished (e.g., the mouse is moved away from section 604) or theadvertisement is finished playing, the rich media content may beginplaying again. Other ways of triggering playing of the advertisement mayalso be appreciated, such as the selection on a touchscreen, input of ashortcut key, double or single selection of window 604, etc.

The advertisement may also include widgets 606 and 608 that allowadditional features to be invoked. For example, widgets 606 and 608provide additional information for the advertisement. The additionalinformation may be provided while the rich media content is playing.This feature may allow the user to view more information about theadvertisement without interrupting the rich media content. Also, therich media content may be paused.

Widget 604, when triggered, may show additional information in a pop-upwindow 610. For example, a user may move the mouse pointer over widget606, which then displays the pop up window 610. In this case, moreinformation is shown for the advertisement. If the user moved the mouseover widget 608, a window may pop up allowing the user to get moreinformation. For example, the user may be able to enter an email addresswhere more information about the advertisement can be sent. Other waysof triggering widgets 606 and 608 may also be appreciated, such as theselection on a touchscreen, input of a shortcut key, the double orsingle selection of widgets 606 and 608, etc.

Other interactive features may also be provided. For example, couponsand maps may be provided. Coupons for the advertisement may be displayedwith the advertisement. A user may select the coupon and be sent to aweb page that allow redemption. Also, selecting the coupon may send theuser the coupon at a user's email address, etc. Including a couponprovides incentive for a user to either view the ad or actually takeaction to purchase a product shown in the ad.

Also, a map may also be included. The map may show local businesses thatare selling products depicted in the advertisement. Also, links to thelocal businesses may be provided. If user's select the links, fees maybe charged to the local businesses by an owner of system 100. The mapsshow an easy way for a user to purchase the product in the ad.

Further, other services may be provided. For example, a link to buytickets for movies may be provided when a movie is being watched.

Although the invention has been described with respect to specificembodiments thereof, these embodiments are merely illustrative, and notrestrictive of the invention. For example, ads can be presented invarious types of association with rich media content. If a first deviceis used to present the rich media content, a separate second device canbe used to display the rich media content.

Any suitable programming language can be used to implement the routinesof embodiments of the present invention including C, C++, Java, assemblylanguage, etc. Different programming techniques can be employed such asprocedural or object oriented. The routines can execute on a singleprocessing device or multiple processors. Although the steps,operations, or computations may be presented in a specific order, thisorder may be changed in different embodiments. In some embodiments,multiple steps shown as sequential in this specification can beperformed at the same time. The sequence of operations described hereincan be interrupted, suspended, or otherwise controlled by anotherprocess, such as an operating system, kernel, etc. The routines canoperate in an operating system environment or as stand-alone routinesoccupying all, or a substantial part, of the system processing.Functions can be performed in hardware, software, or a combination ofboth. Unless otherwise stated, functions may also be performed manually,in whole or in part.

In the description herein, numerous specific details are provided, suchas examples of components and/or methods, to provide a thoroughunderstanding of embodiments of the present invention. One skilled inthe relevant art will recognize, however, that an embodiment of theinvention can be practiced without one or more of the specific details,or with other apparatus, systems, assemblies, methods, components,materials, parts, and/or the like. In other instances, well-knownstructures, materials, or operations are not specifically shown ordescribed in detail to avoid obscuring aspects of embodiments of thepresent invention.

A “computer-readable medium” for purposes of embodiments of the presentinvention may be any medium that can contain, store, communicate,propagate, or transport the program for use by or in connection with theinstruction execution system, apparatus, system or device. The computerreadable medium can be, by way of example only but not by limitation, anelectronic, magnetic, optical, electromagnetic, infrared, orsemiconductor system, apparatus, system, device, propagation medium, orcomputer memory.

Embodiments of the present invention can be implemented in the form ofcontrol logic in software or hardware or a combination of both. Thecontrol logic may be stored in an information storage medium, such as acomputer-readable medium, as a plurality of instructions adapted todirect an information processing device to perform a set of stepsdisclosed in embodiments of the present invention. Based on thedisclosure and teachings provided herein, a person of ordinary skill inthe art will appreciate other ways and/or methods to implement thepresent invention.

A “processor” or “process” includes any human, hardware and/or softwaresystem, mechanism or component that processes data, signals or otherinformation. A processor can include a system with a general-purposecentral processing unit, multiple processing units, dedicated circuitryfor achieving functionality, or other systems. Processing need not belimited to a geographic location, or have temporal limitations. Forexample, a processor can perform its functions in “real time,”“offline,” in a “batch mode,” etc. Portions of processing can beperformed at different times and at different locations, by different(or the same) processing systems.

Reference throughout this specification to “one embodiment”, “anembodiment”, or “a specific embodiment” means that a particular feature,structure, or characteristic described in connection with the embodimentis included in at least one embodiment of the present invention and notnecessarily in all embodiments. Thus, respective appearances of thephrases “in one embodiment”, “in an embodiment”, or “in a specificembodiment” in various places throughout this specification are notnecessarily referring to the same embodiment. Furthermore, theparticular features, structures, or characteristics of any specificembodiment of the present invention may be combined in any suitablemanner with one or more other embodiments. It is to be understood thatother variations and modifications of the embodiments of the presentinvention described and illustrated herein are possible in light of theteachings herein and are to be considered as part of the spirit andscope of the present invention.

Embodiments of the invention may be implemented by using a programmedgeneral purpose digital computer, by using application specificintegrated circuits, programmable logic devices, field programmable gatearrays, optical, chemical, biological, quantum or nanoengineeredsystems, components and mechanisms may be used. In general, thefunctions of embodiments of the present invention can be achieved by anymeans as is known in the art. Distributed, or networked systems,components and circuits can be used. Communication, or transfer, of datamay be wired, wireless, or by any other means.

It will also be appreciated that one or more of the elements depicted inthe drawings/figures can also be implemented in a more separated orintegrated manner, or even removed or rendered as inoperable in certaincases, as is useful in accordance with a particular application. It isalso within the spirit and scope of the present invention to implement aprogram or code that can be stored in a machine-readable medium topermit a computer to perform any of the methods described above.

Additionally, any signal arrows in the drawings/Figures should beconsidered only as exemplary, and not limiting, unless otherwisespecifically noted. Furthermore, the term “or” as used herein isgenerally intended to mean “and/or” unless otherwise indicated.Combinations of components or steps will also be considered as beingnoted, where terminology is foreseen as rendering the ability toseparate or combine is unclear.

As used in the description herein and throughout the claims that follow,“a”, “an” and “the” includes plural references unless the contextclearly dictates otherwise. Also, as used in the description herein andthroughout the claims that follow, the meaning of “in” includes “in” and“on” unless the context clearly dictates otherwise.

The foregoing description of illustrated embodiments of the presentinvention, including what is described in the Abstract, is not intendedto be exhaustive or to limit the invention to the precise formsdisclosed herein. While specific embodiments of, and examples for, theinvention are described herein for illustrative purposes only, variousequivalent modifications are possible within the spirit and scope of thepresent invention, as those skilled in the relevant art will recognizeand appreciate. As indicated, these modifications may be made to thepresent invention in light of the foregoing description of illustratedembodiments of the present invention and are to be included within thespirit and scope of the present invention.

Thus, while the present invention has been described herein withreference to particular embodiments thereof, a latitude of modification,various changes and substitutions are intended in the foregoingdisclosures, and it will be appreciated that in some instances somefeatures of embodiments of the invention will be employed without acorresponding use of other features without departing from the scope andspirit of the invention as set forth. Therefore, many modifications maybe made to adapt a particular situation or material to the essentialscope and spirit of the present invention. It is intended that theinvention not be limited to the particular terms used in followingclaims and/or to the particular embodiment disclosed as the best modecontemplated for carrying out this invention, but that the inventionwill include any and all embodiments and equivalents falling within thescope of the appended claims.

1. A method for serving advertisements, the method comprising:determining when an advertisement correlates to subject matterassociated with a portion of rich media content; determining renderingpreferences associated with the advertisement; and serving theadvertisement to a device, the advertisement being served such that itcan be rendered relative to a time the portion of the rich media contentis being played on the device based on the rendering preferences.
 2. Themethod of claim 1, wherein serving the advertisement comprises servingthe advertisement such that the advertisement is rendered on the deviceserially with the portion of rich media content.
 3. The method of claim2, wherein serially comprises before, after, or in the middle of theplaying of the portion of the rich media content.
 4. The method of claim1, wherein serving the advertisement comprises serving the advertisementsuch that the advertisement is rendered on the device in parallel withthe portion of rich media content.
 5. The method of claim 1, whereinserving the advertisement comprises serving the advertisement such thatthe advertisement is injected into or laid on top of portion of richmedia content.
 6. The method of claim 1, wherein serving theadvertisement comprises causing rendering of the advertisement on thedevice.
 7. The method of claim 1, wherein the advertisement is adifferent form of rich media content than the portion of the rich mediacontent.
 8. The method of claim 1, wherein the rendering preferencesindicate a position to display the advertisement relative to the portionof the rich media content.
 9. The method of claim 1, wherein therendering preferences are specified by a user of the user device, acontent owner of the rich media content and/or the advertiser of theadvertisement.
 10. The method of claim 1, wherein the advertisementincludes a coupon or map related to the advertisement.
 11. The method ofclaim 1, wherein serving the advertisement comprises causing storing theadvertisement at the device.
 12. The method of claim 1, whereindetermining when an advertisement correlates to subject matterassociated with a portion of rich media content comprises usingcorrelation recognition detection techniques to determine that theadvertisement correlates the subject matter.
 13. The method of claim 1,wherein determining when an advertisement correlates to subject matterassociated with a portion of rich media content comprises time aligningthe advertisement to correlate the advertisement to the subject matterto maximize a criteria.
 14. A method for rendering advertisements, themethod comprising: playing rich media content on a device; receiving anadvertisement that correlates to subject matter associated with aportion of the rich media content being played; determining renderingpreferences associated with the advertisement; and rendering theadvertisement, the advertisement being rendered such that it can berendered relative to a time the portion of the rich media content isbeing played on the device based on the rendering preferences.
 15. Themethod of claim 14, wherein the advertisement is rendered on a displayof the device.
 16. The method of claim 14, wherein the advertisement isrendered through a speaker of the device.
 17. The method of claim 16,wherein rendering the advertisement comprises rendering theadvertisement such that the advertisement is rendered serially with theportion of rich media content.
 18. The method of claim 17, whereinserially comprises before or after the playing of the portion of therich media content.
 19. The method of claim 14, wherein theadvertisement is rendered with audio or visual cues that alert the userof a new and/or changing ad placement.
 20. The method of claim 14,wherein rendering the advertisement comprises rendering theadvertisement such that the advertisement is rendered on the device inparallel with the portion of rich media content.
 21. The method of claim14, wherein rendering the advertisement comprises rendering theadvertisement such that the advertisement is injected into the portionof rich media content.
 22. The method of claim 14, further comprising:detecting an event associated with the advertisement; and performing anaction for the event.
 23. The method of claim 22, wherein the actioncomprises: stopping the playing of the rich media content; and startingplaying of the advertisement.
 24. The method of claim 23, where theaction comprises displaying additional information for theadvertisement.
 25. The method of claim 14, further comprising receivingadditional information for the user.
 26. The method of claim 14, whereindetermining when an advertisement correlates to subject matterassociated with a portion of rich media content comprises usingcorrelation recognition detection techniques to determine that theadvertisement correlates the subject matter.
 27. The method of claim 14,wherein determining when an advertisement correlates to subject matterassociated with a portion of rich media content comprises time aligningthe advertisement to correlate the advertisement to the subject matterto maximize a criteria.
 28. An apparatus configured to serveadvertisements, the apparatus comprising: a correlation engineconfigured to determine when an advertisement correlates to subjectmatter associated with a portion of rich media content; a renderingformatter configured to determine rendering preferences associated withthe advertisement; and an ad server configured to serve theadvertisement to a device, the advertisement being served such that itcan be rendered relative to a time the portion of the rich media contentis being played on the device based on the rendering preferences. 29.The apparatus of claim 28, wherein the ad server is configured to servethe advertisement such that the advertisement is rendered on the deviceserially with the portion of rich media content.
 30. The apparatus ofclaim 28, wherein the ad server is configured to serve the advertisementsuch that the advertisement is rendered on the device in parallel withthe portion of rich media content.
 31. The apparatus of claim 28,wherein the ad server is configured to serve the advertisement such thatthe advertisement is injected into the portion of rich media content.32. The apparatus of claim 28, wherein the correlation engine isconfigured to correlate the advertisement to use correlation recognitiondetection techniques to determine that the advertisement correlates thesubject matter.
 33. The apparatus of claim 28, wherein the correlationengine is configured to time align the advertisement to correlate theadvertisement to the subject matter to maximize a criteria.
 34. A methodfor correlating advertisements to rich media content, the methodcomprising: recognizing information for the rich media content;determining a plurality of advertisements to correlate to the recognizedinformation; and selecting an advertisement from the plurality ofadvertisements to associate with a portion of the rich media contentbased on one or more criteria.
 35. The method of claim 34, whereindetermining the plurality of advertisements to correlate comprises:determining match information associated with an advertisement in theplurality of advertisements; determining if the match information isfound in the rich media content; and correlating the advertisement withthe rich media content if the match information is found in the richmedia content.
 36. The method of claim 34, wherein selecting theadvertisement comprises: determining scores for the one or more criteriafor each of the plurality of advertisements; and selecting theadvertisement based on the determined scores.
 37. The method of claim36, wherein the one or more criteria comprises a bid price for theadvertisement, wherein an amount of the bid affects the score given tothe advertisement.
 38. The method of claim 37, wherein selecting theadvertisement is performed to maximize revenue from bids associated withthe plurality of advertisements.
 39. The method of claim 36, wherein theone or more criteria comprise lattice probability, match concentration,match position, geo-location, demographic, behavioral, and psychographicprofile, and/or past performance.